[staging] python313Packages.pyzmq: disable flaky tests #367255
Conversation
wegank
added
the
12.approvals: 1
| @@ -63,6 +63,8 @@ buildPythonPackage rec { | |||
| rm -r zmq | |||
| ''; | |||
|
|
|||
| pytestFlagsArray = [ "-m 'not flaky'" ]; | |||
There was a problem hiding this comment.
Not sure if this will work with structured attributes (#347194); might need pytestFlagsArray+=(-m 'not flaky') in a Bash phase? cc @ShamrockLee
There was a problem hiding this comment.
this is a pretty common pattern in the python modules. tho I can probably write it as [ "-m" "'not flaky'"] which might be more future compatible ?
There was a problem hiding this comment.
yup, that worked got:
collected 364 items / 84 deselected / 1 skipped / 280 selected
which is what it should be when the 3 flaky tests are disabled.
There was a problem hiding this comment.
tho when I add __structuredAttrs = true; both revisions have the same result -- 3 flaky tests are disabled so perhaps it just doesn't matter.
There was a problem hiding this comment.
#347194 will leave pytestFlagsArray as is and introduce pytestFlags as the new, conforming interface. (The pytestFlags version will be pytestFlags = [ "-m" "not flaky" ];, and will require __structuredAttrs = true to keep the spaced argument from splitting.)
Due to historical reasons, pytestFlagsArray are string-concatenated and Bash-evaluated before appending to the command. Unfortunately, specifying it as a Bash array doesn't prevent such Bash-evaluating behaviour.
There was a problem hiding this comment.
I guess it’s fine for now then; migrating off the broken pytestFlagsArray semantics is a task for another time.
paparodeo
force-pushed
the
pyzmq-disable-flaky-tests
branch
from
a37ce89 to
a67e93f
Compare
3 tests are marked as flaky and fail periodically on hydra, disable them. Alternatively, adding pytest-rerunfailures package will enable the tests to be rerun (they are marked as `rerun=3`) but just disable them all together. before: collected 364 items / 81 deselected / 1 skipped / 283 selected after: collected 364 items / 84 deselected / 1 skipped / 280 selected https://github.com/pytest-dev/pytest-rerunfailures https://cache.nixos.org/log/ja0w2jni5qzlgrzp2sjl67rcjzrxxz0r-python3.13-pyzmq-26.2.0.drv
paparodeo
force-pushed
the
pyzmq-disable-flaky-tests
branch
from
a67e93f to
be63334
Compare
wegank
removed
the
12.approvals: 1
3 tests are marked as flaky and fail periodically on hydra, disable them. Alternatively, adding pytest-rerunfailures package will enable the tests to be rerun (they are marked as
rerun=3) but just disable them all together.before:
collected 364 items / 81 deselected / 1 skipped / 283 selected
after:
collected 364 items / 84 deselected / 1 skipped / 280 selected
https://github.com/pytest-dev/pytest-rerunfailures
Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.